[TRTLLM-6496][feat] Add LoRa Torch tests for the latest NIM model list #6806
Conversation
|
Important Review skippedMore than 25% of the files skipped due to max files limit. The review is being skipped to prevent a low-quality review. 193 files out of 300 files are above the max files limit of 100. Please upgrade to Pro plan to get higher limits. You can disable this status message by setting the 📝 WalkthroughWalkthroughRefines head_size extraction in Torch model_config. Adds a common helper and a new Torch multi-LoRA end-to-end test, then extends example tests for Llama, Mistral, Nemotron NAS, and Phi using that helper with CI profiling. Updates fixtures for new model roots, augments H100 test matrix, and adjusts a unit test’s LLM init. Changes
Sequence Diagram(s)sequenceDiagram
participant Tester as pytest test_*.py
participant Helper as defs.common::test_llm_torch_multi_lora_support
participant LLM as LLM_torch
participant LoRA as LoRA Manager
participant Gen as Generation Engine
Tester->>Helper: build prompts, LoraConfig, SamplingParams
Helper->>LLM: initialize(model_dir, lora_config)
loop per prompt
Helper->>LoRA: build LoRARequest (optional per prompt)
end
Helper->>LLM: generate(prompts, sampling_params, lora_requests)
LLM->>Gen: run inference with applied LoRA adapters
Gen-->>LLM: outputs
LLM-->>Helper: texts
Helper-->>Tester: print/return results
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
Suggested reviewers
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (8)
tests/integration/defs/examples/test_phi.py (1)
453-477: Solid Torch multi-LoRA test for Phi-4-mini; minor nit: drop unused fixture.The test is well-scoped and reuses the common Torch helper with correct targets (qkv_proj/attn_qkv).
phi_example_rootisn’t used in this function; consider removing it to keep the signature clean.-def test_phi_4_mini_instruct_with_bf16_lora_torch( - phi_example_root, llm_datasets_root, qcache_dir_without_install_package, - llm_venv, engine_dir, llm_phi_model_root): +def test_phi_4_mini_instruct_with_bf16_lora_torch( + llm_datasets_root, qcache_dir_without_install_package, + llm_venv, engine_dir, llm_phi_model_root):tests/integration/defs/examples/test_llama.py (2)
4076-4078: Address line-length lint (E501) by splitting the print.Static analysis flagged this as >120 chars. Split the message to keep within our style constraints.
- print( - f"test_llm_torch_multi_lora_support: {defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support')} sec" - ) + elapsed = defs.ci_profiler.elapsed_time_in_sec( + "test_llm_torch_multi_lora_support") + print(f"test_llm_torch_multi_lora_support: {elapsed} sec")
4110-4111: Address line-length lint (E501) on the 8-GPU print.Same suggestion as above.
- print( - f"test_llm_torch_multi_lora_support: {defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support')} sec" - ) + elapsed = defs.ci_profiler.elapsed_time_in_sec( + "test_llm_torch_multi_lora_support") + print(f"test_llm_torch_multi_lora_support: {elapsed} sec")tests/integration/defs/examples/test_nemotron_nas.py (5)
129-156: Well-structured test for Nemotron Nano 8B.The test is properly configured with:
- Memory requirements check
- Clear documentation
- CI profiling for performance monitoring
- Appropriate LoRA configuration for the model size
Consider splitting long print statement.
The print statement on line 153-155 exceeds the line length limit.
- print( - f"test_llm_torch_multi_lora_support: {defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support')} sec" - ) + elapsed_time = defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support') + print(f"test_llm_torch_multi_lora_support: {elapsed_time} sec")
147-147: Missingtarget_trtllm_modulesparameter.Unlike the 49B and 253B tests below, this test doesn't specify
target_trtllm_modules. While the default intest_llm_torch_multi_lora_supportis["attn_q", "attn_k", "attn_v"], it would be more consistent to explicitly provide this parameter.target_hf_modules=["q_proj", "k_proj", "v_proj"], + target_trtllm_modules=["attn_q", "attn_k", "attn_v"], zero_lora_weights=True,
158-194: Well-configured test for larger model with appropriate resource requirements.The test correctly:
- Marks itself as skipped for local testing (4 GPUs required)
- Specifies both HF and TRT-LLM module mappings for comprehensive LoRA coverage
- Uses appropriate tensor parallelism for the model size
- Includes MLP modules in addition to attention modules
Consider splitting long print statement.
Similar to the previous test, the print statement exceeds the line length limit.
- print( - f"test_llm_torch_multi_lora_support: {defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support')} sec" - ) + elapsed_time = defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support') + print(f"test_llm_torch_multi_lora_support: {elapsed_time} sec")
196-225: Properly scaled test for the largest model.The test appropriately:
- Requires 8 GPUs with proper skip markers
- Uses maximum tensor parallelism
- Maintains consistent structure with other tests
Consider splitting long print statement.
The print statement exceeds the line length limit.
- print( - f"test_llm_torch_multi_lora_support: {defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support')} sec" - ) + elapsed_time = defs.ci_profiler.elapsed_time_in_sec('test_llm_torch_multi_lora_support') + print(f"test_llm_torch_multi_lora_support: {elapsed_time} sec")
217-217: Missingtarget_trtllm_modulesparameter for consistency.Similar to the Nano 8B test, this Ultra 253B test doesn't specify
target_trtllm_modules. For consistency with the Super 49B test and clarity, consider explicitly providing this parameter.target_hf_modules=["q_proj", "k_proj", "v_proj"], + target_trtllm_modules=["attn_q", "attn_k", "attn_v"], zero_lora_weights=True,
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
tensorrt_llm/_torch/model_config.py(1 hunks)tests/integration/defs/common.py(4 hunks)tests/integration/defs/conftest.py(3 hunks)tests/integration/defs/examples/test_llama.py(2 hunks)tests/integration/defs/examples/test_mistral.py(3 hunks)tests/integration/defs/examples/test_nemotron_nas.py(2 hunks)tests/integration/defs/examples/test_phi.py(2 hunks)tests/integration/test_lists/test-db/l0_h100.yml(2 hunks)tests/unittest/llmapi/test_llm_pytorch.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
**/*.py: Python code should conform to Python 3.8+.
Indent Python code with 4 spaces. Do not use tabs.
Always maintain the namespace when importing in Python, even if only one class or function from a module is used.
Python filenames should use snake_case (e.g., some_file.py).
Python classes should use PascalCase (e.g., class SomeClass).
Python functions and methods should use snake_case (e.g., def my_awesome_function():).
Python local variables should use snake_case. Prefix k for variable names that start with a number (e.g., k_99th_percentile).
Python global variables should use upper snake_case and prefix G (e.g., G_MY_GLOBAL).
Python constants should use upper snake_case (e.g., MY_CONSTANT).
Avoid shadowing variables declared in an outer scope in Python.
Initialize all externally visible members of a Python class in the constructor.
For interfaces that may be used outside a Python file, prefer docstrings over comments.
Comments in Python should be reserved for code within a function, or interfaces that are local to a file.
Use Google style docstrings for Python classes and functions, which can be parsed by Sphinx.
Attributes and variables in Python can be documented inline; attribute docstrings will be rendered under the class docstring.
Avoid using reflection in Python when functionality can be easily achieved without it.
When using try-except blocks in Python, limit the except to the smallest set of errors possible.
When using try-except blocks to handle multiple possible variable types in Python, keep the body of the try as small as possible, using the else block to implement the logic.
Files:
tensorrt_llm/_torch/model_config.pytests/unittest/llmapi/test_llm_pytorch.pytests/integration/defs/examples/test_llama.pytests/integration/defs/conftest.pytests/integration/defs/examples/test_phi.pytests/integration/defs/examples/test_mistral.pytests/integration/defs/common.pytests/integration/defs/examples/test_nemotron_nas.py
**/*.{cpp,h,hpp,cc,cxx,cu,py}
📄 CodeRabbit Inference Engine (CODING_GUIDELINES.md)
All TensorRT-LLM Open Source Software code should contain an NVIDIA copyright header that includes the current year. This includes .cpp, .h, .cu, .py, and any other source files which are compiled or interpreted.
Files:
tensorrt_llm/_torch/model_config.pytests/unittest/llmapi/test_llm_pytorch.pytests/integration/defs/examples/test_llama.pytests/integration/defs/conftest.pytests/integration/defs/examples/test_phi.pytests/integration/defs/examples/test_mistral.pytests/integration/defs/common.pytests/integration/defs/examples/test_nemotron_nas.py
🧠 Learnings (3)
📓 Common learnings
Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.
📚 Learning: 2025-07-28T17:06:08.621Z
Learnt from: moraxu
PR: NVIDIA/TensorRT-LLM#6303
File: tests/integration/test_lists/qa/examples_test_list.txt:494-494
Timestamp: 2025-07-28T17:06:08.621Z
Learning: In TensorRT-LLM testing, it's common to have both CLI flow tests (test_cli_flow.py) and PyTorch API tests (test_llm_api_pytorch.py) for the same model. These serve different purposes: CLI flow tests validate the traditional command-line workflow, while PyTorch API tests validate the newer LLM API backend. Both are legitimate and should coexist.
Applied to files:
tests/integration/defs/examples/test_llama.pytests/integration/defs/examples/test_phi.pytests/integration/defs/examples/test_mistral.pytests/integration/defs/common.pytests/integration/test_lists/test-db/l0_h100.ymltests/integration/defs/examples/test_nemotron_nas.py
📚 Learning: 2025-08-01T15:14:45.673Z
Learnt from: yibinl-nvidia
PR: NVIDIA/TensorRT-LLM#6506
File: examples/models/core/mixtral/requirements.txt:3-3
Timestamp: 2025-08-01T15:14:45.673Z
Learning: In TensorRT-LLM, examples directory can have different dependency versions than the root requirements.txt file. Version conflicts between root and examples dependencies are acceptable because examples are designed to be standalone and self-contained.
Applied to files:
tests/integration/defs/common.py
🧬 Code Graph Analysis (7)
tensorrt_llm/_torch/model_config.py (2)
tests/unittest/_torch/test_resource_manager.py (1)
head_size(79-80)tensorrt_llm/logger.py (1)
warning(131-132)
tests/unittest/llmapi/test_llm_pytorch.py (6)
tests/unittest/_torch/test_beam_search.py (1)
llm(37-49)tests/unittest/_torch/test_best_of_n.py (1)
llm(36-42)tensorrt_llm/llmapi/llm.py (1)
LLM(1080-1096)tensorrt_llm/_torch/llm.py (1)
LLM(4-9)tensorrt_llm/llmapi/llm_args.py (2)
model_dir(1089-1091)model_dir(1094-1098)tensorrt_llm/_torch/models/modeling_phi4mm.py (1)
lora_config(600-620)
tests/integration/defs/examples/test_llama.py (2)
tests/integration/defs/common.py (1)
test_llm_torch_multi_lora_support(884-976)tests/integration/defs/conftest.py (6)
llama_example_root(243-254)llm_datasets_root(191-192)qcache_dir_without_install_package(1763-1780)llm_venv(699-715)engine_dir(1693-1708)llama_model_root(956-1044)
tests/integration/defs/conftest.py (1)
tests/integration/defs/triton_server/conftest.py (1)
models_root(273-277)
tests/integration/defs/examples/test_phi.py (1)
tests/integration/defs/conftest.py (5)
llm_datasets_root(191-192)qcache_dir_without_install_package(1763-1780)llm_venv(699-715)engine_dir(1693-1708)llm_phi_model_root(1426-1442)
tests/integration/defs/examples/test_mistral.py (2)
tests/integration/defs/common.py (3)
convert_weights(166-525)quantize_data(578-628)test_llm_torch_multi_lora_support(884-976)tests/integration/defs/conftest.py (6)
llama_example_root(243-254)llm_datasets_root(191-192)qcache_dir_without_install_package(1763-1780)llm_venv(699-715)engine_dir(1693-1708)llm_mistral_model_root(1361-1376)
tests/integration/defs/common.py (4)
tensorrt_llm/executor/request.py (1)
LoRARequest(24-53)tensorrt_llm/sampling_params.py (1)
SamplingParams(125-486)tests/integration/defs/conftest.py (1)
llm_venv(699-715)tensorrt_llm/_torch/models/modeling_phi4mm.py (2)
lora_config(600-620)lora_request(623-644)
🪛 Ruff (0.12.2)
tests/integration/defs/examples/test_llama.py
4068-4068: Line too long (125 > 120)
(E501)
4100-4100: Line too long (125 > 120)
(E501)
tests/integration/defs/examples/test_nemotron_nas.py
153-153: Line too long (125 > 120)
(E501)
190-190: Line too long (125 > 120)
(E501)
221-221: Line too long (125 > 120)
(E501)
🔇 Additional comments (17)
tests/integration/test_lists/test-db/l0_h100.yml (2)
92-93: Matrix expansion for Torch BF16-LoRA looks good.Entries align with added Torch multi-LoRA tests and existing fixtures. No YAML issues spotted.
220-221: Post-merge Torch BF16-LoRA additions are consistent.Tests are correctly scoped and map to newly added helpers. Good to go.
tests/integration/defs/conftest.py (2)
1024-1048: New llama 3.x instruct variants mapping is correct.The fixture resolves the new labels to expected paths and asserts presence under LLM_MODELS_ROOT. This unblocks newly added Torch tests.
1376-1379: Incorrect reference to hard-coded path in integration conftest.pyI couldn’t find any
/code/tensorrt_llm/my_hf_models/...literal in
tests/integration/defs/conftest.py– all model roots there use
llm_models_root()+os.path.join(...). The review comment appears to
target a snippet that no longer exists. Please ignore this suggestion.Likely an incorrect or invalid review comment.
tests/integration/defs/examples/test_llama.py (3)
29-31: Importing the Torch multi-LoRA helper is appropriate here.Good reuse of the new shared helper to avoid duplication across examples.
4043-4075: Nice addition of Llama 3.x Torch multi-LoRA coverage.
- Proper skip guards and parametrization.
- Reuses get_test_prompts and common helper, consistent with team patterns. Referencing the retrieved learning: having both CLI-flow and PyTorch-API tests is expected and beneficial for coverage.
4081-4112: 8-GPU Torch variant stub looks good; skipping is fine until infra is ready.The test mirrors the 1-GPU flow and correctly bumps tensor_parallel_size to 8. Once hardware is available, drop the skip marker to enable.
Ensure fixtures and model paths for 'llama-3.3-70b-instruct' are present in LLM_MODELS_ROOT before unskipping.
tests/integration/defs/examples/test_mistral.py (5)
18-18: LGTM!The addition of the
defs.ci_profilerimport is appropriate for the profiling functionality used in the new test functions.
22-22: LGTM!The import of
test_llm_torch_multi_lora_supportaligns with its usage in the new Torch-based LoRA tests.
49-66: Reasonable to comment out Windows-specific fixture.The fixture appears to be intended for installing flash-attn on non-Windows systems, which makes sense given that flash-attn doesn't have Windows wheels. Since the new tests don't appear to require this dependency, commenting it out is acceptable.
301-324: LGTM! Well-structured LoRA test for Mistral-7b-v0.1.The test is well-organized with appropriate:
- Skip markers for GPU compatibility (
skip_pre_ada) and memory requirements- Clear documentation via docstring
- Proper use of CI profiling for performance monitoring
- Sensible default parameters for LoRA testing
327-352: LGTM! Well-configured multi-GPU LoRA test.The test properly handles:
- Multi-GPU requirements with
skip_less_device(2)and memory checks- Tensor parallelism with
tensor_parallel_size=2- Proper fixture parameterization for the specific model
- CI profiling for performance tracking
tests/integration/defs/common.py (4)
25-29: LGTM! Appropriate imports for Torch-based LoRA testing.The imports and aliases are correctly added to support the new LLM-API Torch backend functionality, maintaining consistency with TensorRT-LLM's naming conventions.
769-796: LGTM! Well-designed helper function for test prompts.The function provides a clean separation between code-related and general prompts, making the tests more maintainable and readable. The prompts are diverse and appropriate for testing language model capabilities.
852-852: LGTM! Good refactoring to use the helper function.Replacing the inline prompt list with a call to
get_test_promptsimproves code maintainability and consistency across tests.
887-980: Comprehensive and well-structured Torch-based LoRA test function.The implementation is thorough with:
- Clear timing instrumentation for performance analysis
- Proper resource management using context manager for LLM lifecycle
- Flexible LoRA request configuration (some with, some without LoRA)
- Detailed output logging for debugging
- Consistent parameter handling with the existing TRT-based test
tests/integration/defs/examples/test_nemotron_nas.py (1)
3-6: LGTM! Appropriate imports for profiling and LoRA testing.The imports correctly add CI profiling support and the Torch-based LoRA test helper function.
moraxu
force-pushed
the
dev-mguzek-add-lora-llmapi-torch-tests
branch
from
d78b984 to
df7e14d
Compare
|
/bot run |
|
PR_Github #15473 [ run ] triggered by Bot |
|
PR_Github #15473 [ run ] completed with state |
|
/bot run |
|
PR_Github #15479 [ run ] triggered by Bot |
|
/bot kill |
|
PR_Github #15481 [ kill ] triggered by Bot |
|
PR_Github #15479 [ run ] completed with state |
|
PR_Github #15481 [ kill ] completed with state |
|
/bot run |
|
PR_Github #15484 [ run ] triggered by Bot |
|
PR_Github #15484 [ run ] completed with state |
moraxu
force-pushed
the
dev-mguzek-add-lora-llmapi-torch-tests
branch
from
6ccaa41 to
4c899ad
Compare
|
/bot run |
|
PR_Github #15637 [ run ] triggered by Bot |
|
PR_Github #15637 [ run ] completed with state |
|
/bot run |
|
PR_Github #15644 [ run ] triggered by Bot |
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
Signed-off-by: Michal Guzek <[email protected]>
moraxu
force-pushed
the
dev-mguzek-add-lora-llmapi-torch-tests
branch
from
dd886af to
4e07bd6
Compare
|
/bot run |
|
PR_Github #20512 [ run ] triggered by Bot |
|
PR_Github #20512 [ run ] completed with state |
|
/bot run |
|
PR_Github #20544 [ run ] triggered by Bot |
|
PR_Github #20544 [ run ] completed with state |
7 participants
Description
[TRTLLM-6496][feat] Add LoRa Torch tests for the latest NIM model list
Tested:
1 GPU:
pytest -sv tests/integration/defs/examples/test_llama.py::test_llama_3_x_with_bf16_lora_torchpytest -sv tests/integration/defs/examples/test_nemotron_nas.py::test_nemotron_nano_8b_lora_torchpytest -sv tests/integration/defs/examples/test_mistral.py::test_mistral_with_bf16_lora_torch2 GPUs:
pytest -sv tests/integration/defs/examples/test_mistral.py::test_mistral_with_bf16_lora_torch4 GPUs:
pytest -sv tests/integration/defs/examples/test_nemotron_nas.py::test_nemotron_super_49b_real_lora_torch8 GPUs:
pytest -sv tests/integration/defs/examples/test_llama.py::test_llama_3_x_with_bf16_lora_torchBlockers:
1 GPU:
pytest -sv tests/integration/defs/examples/test_phi.py::test_phi_4_mini_instruct_with_bf16_lora_torch- an unresolved import issue: https://huggingface.co/microsoft/Phi-4-mini-instruct/discussions/398 GPUs:
pytest -sv tests/integration/defs/examples/test_nemotron_nas.py::test_nemotron_ultra_253b_lora_torch- OOMTest Coverage
GitHub Bot Help
/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...Provide a user friendly way for developers to interact with a Jenkins server.
Run
/bot [-h|--help]to print this help message.See details below for each supported subcommand.
run [--reuse-test (optional)pipeline-id --disable-fail-fast --skip-test --stage-list "A10-PyTorch-1, xxx" --gpu-type "A30, H100_PCIe" --test-backend "pytorch, cpp" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx" --detailed-log --debug(experimental)]Launch build/test pipelines. All previously running jobs will be killed.
--reuse-test (optional)pipeline-id(OPTIONAL) : Allow the new pipeline to reuse build artifacts and skip successful test stages from a specified pipeline or the last pipeline if no pipeline-id is indicated. If the Git commit ID has changed, this option will be always ignored. The DEFAULT behavior of the bot is to reuse build artifacts and successful test results from the last pipeline.--disable-reuse-test(OPTIONAL) : Explicitly prevent the pipeline from reusing build artifacts and skipping successful test stages from a previous pipeline. Ensure that all builds and tests are run regardless of previous successes.--disable-fail-fast(OPTIONAL) : Disable fail fast on build/tests/infra failures.--skip-test(OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.--stage-list "A10-PyTorch-1, xxx"(OPTIONAL) : Only run the specified test stages. Examples: "A10-PyTorch-1, xxx". Note: Does NOT update GitHub check status.--gpu-type "A30, H100_PCIe"(OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.--test-backend "pytorch, cpp"(OPTIONAL) : Skip test stages which don't match the specified backends. Only support [pytorch, cpp, tensorrt, triton]. Examples: "pytorch, cpp" (does not run test stages with tensorrt or triton backend). Note: Does NOT update GitHub pipeline status.--only-multi-gpu-test(OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.--disable-multi-gpu-test(OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.--add-multi-gpu-test(OPTIONAL) : Force run the multi-GPU tests in addition to running L0 pre-merge pipeline.--post-merge(OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.--extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx"(OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-TensorRT-Post-Merge-1, xxx".--detailed-log(OPTIONAL) : Enable flushing out all logs to the Jenkins console. This will significantly increase the log volume and may slow down the job.--debug(OPTIONAL) : Experimental feature. Enable access to the CI container for debugging purpose. Note: Specify exactly one stage in thestage-listparameter to access the appropriate container environment. Note: Does NOT update GitHub check status.For guidance on mapping tests to stage names, see
docs/source/reference/ci-overview.mdand the
scripts/test_to_stage_mapping.pyhelper.kill
killKill all running builds associated with pull request.
skip
skip --comment COMMENTSkip testing for latest commit on pull request.
--comment "Reason for skipping build/test"is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.reuse-pipeline
reuse-pipelineReuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.
Summary by CodeRabbit